Integrating musicological knowledge into a probabilistic framework for chord and key extraction

In this contribution a formerly developed probabilistic framework for the simultaneous detection of chords and keys in polyphonic audio is further extended and validated. The system behaviour is controlled by a small set of carefully defined free parameters. This has permitted us to conduct an experimental study which sheds a new light on the importance of musicological knowledge in the context of chord extraction. Some of the obtained results are at least surprising and, to our knowledge, never reported as such before

Exploiting prior knowledge during automatic key and chord estimation from musical audio

Chords and keys are two ways of describing music. They are exemplary of a general class of symbolic notations that musicians use to exchange information about a music piece. This information can range from simple tempo indications such as “allegro” to precise instructions for a performer of the music. Concretely, both keys and chords are timed labels that describe the harmony during certain time intervals, where harmony refers to the way music notes sound together. Chords describe the local harmony, whereas keys offer a more global overview and consequently cover a sequence of multiple chords. Common to all music notations is that certain characteristics of the music are described while others are ignored. The adopted level of detail depends on the purpose of the intended information exchange. A simple description such as “menuet”, for example, only serves to roughly describe the character of a music piece. Sheet music on the other hand contains precise information about the pitch, discretised information pertaining to timing and limited information about the timbre. Its goal is to permit a performer to recreate the music piece. Even so, the information about timing and timbre still leaves some space for interpretation by the performer. The opposite of a symbolic notation is a music recording. It stores the music in a way that allows for a perfect reproduction. The disadvantage of a music recording is that it does not allow to manipulate a single aspect of a music piece in isolation, or at least not without degrading the quality of the reproduction. For instance, it is not possible to change the instrumentation in a music recording, even though this would only require the simple change of a few symbols in a symbolic notation. Despite the fundamental differences between a music recording and a symbolic notation, the two are of course intertwined. Trained musicians can listen to a music recording (or live music) and write down a symbolic notation of the played piece. This skill allows one, in theory, to create a symbolic notation for each recording in a music collection. In practice however, this would be too labour intensive for the large collections that are available these days through online stores or streaming services. Automating the notation process is therefore a necessity, and this is exactly the subject of this thesis. More specifically, this thesis deals with the extraction of keys and chords from a music recording. A database with keys and chords opens up applications that are not possible with a database of music recordings alone. On one hand, chords can be used on their own as a compact representation of a music piece, for example to learn how to play an accompaniment for singing. On the other hand, keys and chords can also be used indirectly to accomplish another goal, such as finding similar pieces. Because music theory has been studied for centuries, a great body of knowledge about keys and chords is available. It is known that consecutive keys and chords form sequences that are all but random. People happen to have certain expectations that must be fulfilled in order to experience music as pleasant. Keys and chords are also strongly intertwined, as a given key implies that certain chords will likely occur and a set of given chords implies an encompassing key in return. Consequently, a substantial part of this thesis is concerned with the question whether musicological knowledge can be embedded in a technical framework in such a way that it helps to improve the automatic recognition of keys and chords. The technical framework adopted in this thesis is built around a hidden Markov model (HMM). This facilitates an easy separation of the different aspects involved in the automatic recognition of keys and chords. Most experiments reviewed in the thesis focus on taking into account musicological knowledge about the musical context and about the expected chord duration. Technically speaking, this involves a manipulation of the transition probabilities in the HMMs. To account for the interaction between keys and chords, every HMM state is actually representing the combination of a key and a chord label. In the first part of the thesis, a number of alternatives for modelling the context are proposed. In particular, separate key change and chord change models are defined such that they closely mirror the way musicians conceive harmony. Multiple variants are considered that differ in the size of the context that is accounted for and in the knowledge source from which they were compiled. Some models are derived from a music corpus with key and chord notations whereas others follow directly from music theory. In the second part of the thesis, the contextual models are embedded in a system for automatic key and chord estimation. The features used in that system are so-called chroma profiles, which represent the saliences of the pitch classes in the audio signal. These chroma profiles are acoustically modelled by means of templates (idealised profiles) and a distance measure. In addition to these acoustic models and the contextual models developed in the first part, durational models are also required. The latter ensure that the chord and key estimations attain specified mean durations. The resulting system is then used to conduct experiments that provide more insight into how each system component contributes to the ultimate key and chord output quality. During the experimental study, the system complexity gets gradually increased, starting from a system containing only an acoustic model of the features that gets subsequently extended, first with duration models and afterwards with contextual models. The experiments show that taking into account the mean key and mean chord duration is essential to arrive at acceptable results for both key and chord estimation. The effect of using contextual information, however, is highly variable. On one hand, the chord change model has only a limited positive impact on the chord estimation accuracy (two to three percentage points), but this impact is fairly stable across different model variants. On the other hand, the chord change model has a much larger potential to improve the key output quality (up to seventeen percentage points), but only on the condition that the variant of the model is well adapted to the tested music material. Lastly, the key change model has only a negligible influence on the system performance. In the final part of this thesis, a couple of extensions to the formerly presented system are proposed and assessed. First, the global mean chord duration is replaced by key-chord specific values, which has a positive effect on the key estimation performance. Next, the HMM system is modified such that the prior chord duration distribution is no longer a geometric distribution but one that better approximates the observed durations in an appropriate data set. This modification leads to a small improvement of the chord estimation performance, but of course, it requires the availability of a suitable data set with chord notations from which to retrieve a target durational distribution. A final experiment demonstrates that increasing the scope of the contextual model only leads to statistically insignificant improvements. On top of that, the required computational load increases greatly

Modeling musicological information as trigrams in a system for simultaneous chord and local key extraction

In this paper, we discuss the introduction of a trigram musicological model in a simultaneous chord and local key extraction system. By enlarging the context of the musicological model, we hoped to achieve a higher accuracy that could justify the associated higher complexity and computational load of the search for the optimal solution. Experiments on multiple data sets have demonstrated that the trigram model has indeed a larger predictive power (a lower perplexity). This raised predictive power resulted in an improvement in the key extraction capabilities, but no improvement in chord extraction when compared to a system with a bigram musicological model

A novel chroma representation of polyphonic music based on multiple pitch tracking techniques

It is common practice to map the frequency content of music onto a chroma representation, but there exist many different schemes for constructing such a representation. In this paper, a new scheme is proposed. It comprises a detection of salient frequencies, a conversion of salient frequencies to notes, a psychophysically motivated weighting of harmonics in support of a note, a restriction of harmonic relations between different notes and a restriction of the deviations from a predefined pitch scale (e.g. the equally tempered western scale). A large-scale experimental evaluation has confirmed that the novel chroma representation more closely matches manual chord labels than the representations generated by six other tested schemes. Therefore, the new chroma representation is expected to improve applications such as song similarity matching and chord detection and labeling

Improving the key extraction performance of a simultaneous local key and chord estimation system

In this paper, significant improvements of a previously developed key and chord extraction system are proposed. The major improvement is the introduction of a separate acoustic model, designed to verify local key hypotheses. The conducted experimental evaluation shows that the presented system improves the state of the art in local key estimation. Our experimental study further demonstrates that the chord estimation performance is already quite robust, whereas the key estimation performance still happens to be sensitive to a number of factors. In particular, we present figures that illustrate the significant impact of the embedded musicological model and the duration of the processed excerpt on the key estimation accuracy

The retinal relaxing factor : update on an enigmatic regulator of the retinal circulation

The retinal circulation is regulated by different local factors and might include the retinal relaxing factor (RRF). This factor is found to be continuously released by the retina and relaxes smooth muscle cells. This review describes the current knowledge about the RRF. Despite many research efforts, the cellular source, identity, mechanism, and physiological role of the RRF remain largely unknown. Thus far, it seems that the RRF is a hydrophilic, thermostable, diffusible chemical messenger, which characteristics do not correspond with most well-known endogenous vasorelaxants. The RRF-induced relaxation seems to rely on activation of the inward rectifier K+ channels and the Rho kinase Ca2+ sensitization mechanism. Voltage-dependent K+ channels and plasma membrane Ca2+-ATPase might also be involved, whereas the involvement of cyclooxygenase is still a point of discussion. Furthermore, it appears that the RRF is involved in other relaxation pathways, namely those of hypoxia, adenosine, and adenosine triphosphate, hydrogen sulfide, c-aminobutyric acid, and dorzolamide

Vasorelaxing effect of resveratrol on bovine retinal arteries

PURPOSE: Resveratrol is a red wine polyphenol that causes vasorelaxation, which could be of interest in the treatment or prevention of eye diseases with an impaired blood flow. In this study, the vasorelaxant capacity of resveratrol (cis and trans) on bovine retinal arteries, its vasorelaxing mechanism, and its influence on the relaxation induced by the retinal relaxing factor (RRF) were examined. METHODS: Isolated bovine retinal arteries were mounted into wire myographs for isometric tension measurements. Concentration-response curves of cis- and trans-resveratrol and concentration-response curves of resveratrol in the absence or presence of the endothelium or different inhibitors were constructed. Relaxations elicited by the RRF with and without resveratrol incubation were also compared. RESULTS: Both resveratrol isomers caused a similar strong concentration-dependent relaxation. Removal of the endothelium or blocking endothelium-dependent pathways did not change the relaxation. Also, K+ channel blockers did not reduce the relaxation, except the 120 mM K+ Krebs Ringer bicarbonate solution. Phorbol 12-myristate 13-acetate and phorbol 12,13-dibutyrate blocked the relaxation partially and so did the inhibition of heme oxygenase-1. Blocking adenylyl cyclase, AMP-activated protein kinase, estrogen receptors, sirtuin 1, or sarco/endoplasmic reticulum Ca2+ ATPase did not have an effect. The relaxation caused by the RRF was not altered by resveratrol incubation. CONCLUSIONS: Cis- and trans-resveratrol relax bovine retinal arteries similarly and concentration dependently. The main relaxation mechanism remains unclear, but K+ channels, carbon monoxide, and the myosin phosphatase pathway may be involved. Resveratrol does not have an influence on the RRF